1. Field
Embodiments of the present invention generally relate to vision systems, and, more particularly, to a method and apparatus for optimizing image quality based on scene content.
2. Description of the Related Art
Digital cameras, camcorders, digital night vision goggles, thermal weapon sights, robotic image sensors and the like contain electronic image sensors to capture light for processing into still or video images of a scene. Image sensors have a pixel array which consists of a multitude of discrete elements of an image called pixels. Each pixel of the image sensor pixel array provides an output level based on the amount of light, heat or other signals incident on the pixel. One primary type of electronic image sensor is a complementary metal oxide semiconductor (CMOS) sensor. CMOS sensors are becoming very commonly used as electronic image sensors because they produce reasonable image quality with reasonable power consumption at a relatively low cost. Additionally, CMOS fabrication processes are used to integrate image processing circuitry alongside the sensor pixel arrays, i.e., a sensor-image processor integrated circuit.
The image sensor in a digital camera or camcorder can generate a noisy, blurred image due to fast motion caused by camera movement or objects in motion in the scene. Image quality is characterized by such factors as signal/noise (strength of the image signal relative to noise of the imaging process), sharpness (absence of focus or motion blur) and dynamic range (range of light levels accurately represented). In traditional imaging there is a natural tradeoff between noise and blur. To reduce blur, the frame integration time is reduced, but this reduces the light captured and as a result the signal strength relative to the noise. Recently a method for recovering signal strength has been proposed in which the sensor is operated at higher frame rates, and frames are then aligned and combined in the digital domain, to form enhanced quality output frames as described in patent application Ser. No. 11/852,632 filed Sep. 10, 2007 which is hereby incorporated by reference in its entirety. This method, called motion adaptive signal integration, in effect provides signal integration in the digital domain, rather than on the sensor array, to achieve both good output signal/noise performance and low motion blur. However increasing sensor frame capture rates comes at the expense of more complex pixel circuitry, poorer sensor image quality and higher power consumption.
In addition, image sensors in general and CMOS image sensors in particular have variations in response from pixel to pixel. The variation in response includes both offset and gain variations. This is called fixed pattern noise. Fixed pattern noise can be mitigated to a certain degree by calibration and correction. Offset and gain levels per pixel can be measured in a calibration period, and then used in a non-uniformity correction (or NUC) circuit to correct the image. However, the residual fixed pattern noise, even after non-uniformity correction may still be apparent and is especially relevant in extreme low light conditions such as when using night vision where the sensor must be very efficient in converting a small amount of incident light to distinguishable output levels.
Therefore, there is a need in the art for a method and apparatus for optimizing image quality while minimizing power, based on image content when using an image sensor, such as a CMOS image sensor.